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ABSTRACT 

The sampling of teachers for nationwide surveys 
offers a challenging endeavor in obtaining a representative and 
adequate sample to truly represent opinions of the teachers. Ten 
national surveys of public school teachers conducted betwee*" 1980 and 
1985 are presented with respect to their sampling design and 
procedures. Concepts and theoretical considerations provide the 
background of the critical review. This paper discusses adequacy ^^nd 
representativeness as criteria of a good sample, sample design, bias, 
sampling error, sampling frame and unit, multistage and stratified 
sampling, and sample size allocation. Problems associated with 
prevailing sampling practices include: (1) lack of sampling frame or 
list of all public school teachers from which a sample itay be drawn; 
(2) the question of assurance that the sample drawn is representative 
of the target population of public school teachers; (?) tl^. 
theoretical connection between a sample design and the computation of 
estimates from the tesultsf (4) the recognition and knowledge of the 
quantitative relationship between the sample design and the precision 
of estimates; and (5) reporting results that demonstrate validity, 
and willingness and ability to sort out the strengths and weaknesses 
of the survey. Solutions to these interrelated problems are 
discussed. (PN) 
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The Educational Reform movement has thrust education 
into the national spotlight resulting in an onslaught of 
innovation in education. One facet of this increased 
attention in education is a surge in the number of national 
surveys of teachers (such as those reviewed in this paper). 
Basicall/i national and state level policy makers are at 
times using information from surveys as component of the 
policy-making process. One major concern however, is 
whether reported results of teacher surveys truly represent 
the opinions of the teachers. This concern relates to the 
representativeness and adequacy of the sample and the 
application of appropriate sampling techniques and design. 

The sampling of teachers for nationwide surveys offers 
one of the most challenging endeavors in obtaining a 
representative and adequate sample and in designing surveys 
that will yield a high degree of precision of estimates. 
The ' inhei^'ent problems in sampling teachers are the 
availability of home telephone numbers i the difficulty of 
accessing teachers through the dirtricts and the absence of 
nationwide reliable data sets froii which samples may be 
drawn. 

PURPOSE AND OBJECTIVES 

The purpose of this study was to examine critically the 
practices and procedures of sampling in national surveys of 
teachers conducted by at least five organizations which 
reported or published results in 1980 to 1985. More 
specif icallyi the objectives were: 

(1) To conduct a critical review of the sampling 
techniques and sampling designs of five or more 
national surveys of teachers: 

(2) To identify pragmatic, procedural, and theoreti- 
cal problems associated with the sampling practi* 
ces prevailing in published teacher surveys; 

(3) To explore the implications of good and bad 
samples to educational policies and decisions; 
and 

(4) To generate solutions and recommendatons that 
will improve the representativeness and adequacy 
of teacher samples i and thereby increase accuracy 
and precision of estimates. 

THEORETICAL CONSIDERATIONS 



Adequacy of the sample and design has direct bearing on 
the precision of estimates. Precision is measured by the 
sampling error which is a function of the variance of 
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estimate of the population parameter (say, mean or 
proportion )whlch in turn is a function of the sample size. 
Thus I the variance of the sample proportion p is 

KX) = ga (1- n) 

where <} = l*Pi and 1-^ is the finite population correction 
(f.p.c). The standard error, Sp the square root of the 
right handside of equation (1), is 



(2; Sp » ^p. 
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Note thatf in sampling from a large population such as in 
national teacher surveys, n/N would be very small, making 
f.p.c. very close to one. 

So for purposes of discussion i let us drop the f.p.c. and 
use 

(3) Sp 

Now, the sampling error is the amount subtracted from, 
and added to the sample proportion, to make an interval 
estimate. This amount is multiplied by a factor z, the 

I value of which is associated 
with the confidence level. Three confidence levels and 
their associated z's are given below: 

Confidence Level z 

90% 1.65 
95% 1.96 
99% 2.58 

The interval estimate of the population proportion is 
usually in this fozm: 

(4) p-zSp P <r pfzSp • 

We can see in formula (3) that the larger our sample 
size n, the smaller Sp becomes; and the smaller Sp is — 
applied to equation (%) — the smaller the interval. The 
smaller this interval is, the higher the precision of 
estimate. Thusi a large saunple increases the precision of 
estimate. The question then is how large a sample should 
O , one have? From equation (3) we can solve for n and have 
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To find n we must have values for Sp and p. This is 
where speculation begins. Suppose I want a precision of 
estimate, of + 3%, It means that in formula (4) I want zS 
to be equal to .03. But z is determined by the confidence^ 
level desired. So, at the 95% confidence level z = 1.96, 
Hence 

zSp = .03 or 

1.96Sp = .03 ; Sp = .0153. 

To ensure that the sample is '"adequate" we want the 
numerator as large as possible. 

Now the product pq, or p(l-p), for combinations of p 
and q such that P + q = 1-0, is largest when p = 0,5. So if 
we use this value, we are assured that the sample size n 
would result in a sampling error no higher than 3%. So wo 
substitute p = 0.5 and = ,0153 in equation (5) and 
obtain 



n - 0.5 X 0.5 

(.0153)^ ' 



n = 1068. 

Thus n = 1068 is an "adequate" sample since it 
satisfies the maximum sampling error tolerable with the 
degree of confidence de*?ired. It is presumed that these 
formulas and procedure are well known to most of the 
participants in the session, and tables for sample sizes are 
available in some books. However, this discussion reminds 
us that there is an explicit procedure to determine exactly 
how large n should be to be adequate. 

Attachment 1 (How to Determine the Sample Size...) is 
supplied as a handy tool for a quick estimation of n. Our 
result n = 1068 can also be confirmed by using McCall's 
(1980) Table H7 (Confidence Level, 95%), p. 354, for a 
population size of 2,000,000. 

The design of the sample » on the other hand takes into 
consideration practical problems — such as the feasibility 
and efficiency of reaching the sampling units (e.g., the 
teachers), the cost, and, of course the precision of 
estimates. Stratified and multi-stage sampling, or a 
combination of these two designs are examples. These 
designs are discussed in the latter part of this paper. 
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Another important consideration, is the representa- 
tiveness of the sample. The representativeness of the 
sample affects the degree of accuracy of estimates. A truly 
random sample is assumed to be representative of the 
population from which it is drawn. However, it is a fact 
that in many national surveys, representativeness should 
become a matter of concern since most surveys depend on a 
list of teachers which is not complete or up to date, for 
the drawing of a sauple. In :fact, a complete lis^ is not 
available. 

The key issue in this pape^ is that the measure of 
precision would be meaningless if applied to a sample that 
is not representative of the population. A secondary issue 
is that in cases where the sample was representative, the 
design could still fall short of optimizing the precision of 
estimates within certain constraints such as cost. 

Representativeness and adequacy are then the ^^o basic 
criteria of a good sampling procedure (Deming, 1966) . To 
get to the fundamental concepts behind these criteria, we 
need to have a clear notion of a few concepts and recognize 
the basic problems of sampling. We need to be able to 
answer such questions as: when is a sample adequate? What 
factors affect the representativeness of a sample? 

The sections that follow intend to clarify these 
concepts and problems. 

Bias and Sampling Error 

The sample can be designed to achieve any desired 
degree of precision. One may wish to estimate a proportion 
or a mean within 1% or 2% of what would be obtained had the 
same procedure been applied to the total population. This 
precision of estimate should not be confused with the 
validity of results for purposes of prediction. There is a 
point in conducting polling surveys to forecast election 
results, as close as possible to the voting time. It should 
not also be confused with inaccuracies resulting from bias 
due to non^response and inaccuracies due to nonsampling 
errors • 

Bias results from lack of randomness, e.g., due to 
incomplete returns or to oversampling one group of 
respondents, say women. Sampling error under conditions of 
randomness is a measure of the degree of tolerance of 
departure of the results from the central measure, with an 
associated probability. Thus, the smaller the sampling 
error, the higher the precision. The four possible 
combinations of the degrees of bias and sampling error are 
illustrated in Attachment 2. 
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The generalization of findings from the sample to the 
population is legitimate only in as far as the sample is 
properly drawn. This is why the sampling design or plan is 
very important. In national surveys of public school 
teachers, the initinl critical problem is the source of 
sampling frame . The frame is merely a list of sampling 
units (e.g. public school teachers) representing the 
population being studied (Farel, 1978). 

In national surveys of public school teachers , the 
population is approximately 2.4 million teachers in the 
United States. In samplingi a distinction, however , is made 
between two definitions of 'population. * 

1. The target population . This is the population for 
which representative information is desired. Thus> the 
target population in this discussion is the 2.4 million 
public school teachers in the United States. 

2. The sampling population . This is the population 
from which a sample will actually be drawn as determined by 
the sampling frame. The sampling frame is the list of 
sampling units (e.g. » teachers, or in multistage sampling, 
schoo] systems, schools). If one wishes to know the 
teachers' opinion on competency testing, the target 
populat5.on could be all K-12 teachers teaching in September 
1985. However, if a list was created, and, for various 
reasons some of these teachers were left out, the remaining 
teachers would be the samp^.ing population . The list created 
from this sampling population is the sampling frame. A 
teacher whose name is in the list (frame) is then a sampling 
unit . 

As we will see later, the sampling frame used by most 
well known survey organizations is the list compiled by 
Market Date Retrieval (MDR) of Westport, Connecticut. 
According to one source in this study, MDR has the most 
complete list but it includes only about 68% of the 2.4 
million teachers. Further, there is no way to get a full 
list of teachers. 

The acceptability of using the sample result to give 
information about the target population should depend on the 
reliability of the sampling frame. It follows then that 
some kind of evaluation of this list is necessary to 
determine whether it is, in fact, representative of the 
total population. 

Multistage and Stratified Sampling 

Some surveys critically reviewed for this paper used 
multistage sampling at two or three stages, instead of using 
the MDR list. Because a list of school systems and a list 
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of schools within the system are available, the sampling 
frana consists of the lists of school systems across the 
different strata. The sampling unit is the school system. 
Stratified sampling may also be incorporated in the design. 

One example is the stratification of school districts 
according to enrollment size. One of the organizations . 
survayed used the stratification shown in Table 1. 

Table 1. Stratification of School Districts 
by Enrollment Size 



Strata No. of 

( enrollment size) School Districts 

100 » 000 and over 22 

50,000-99,999 47 

25,000-49, 999 118 

12,000-24,999 347 

6,000-11,999 926 

3,000- 5,999 1,856 

1,200- 2,999 3,475 

300- 1,199 4,836 

100- 299 4.004 

Total 15,631 

A discussion of the related sampling procedure is in 
the section on the critical review of national surveys 
conducted these past five years. ^ 

The rationale behind the multistage sampling of public 
school teachers, is that in the absence of a reliable 
sampling frame, the teachers could be sampled (reached) 
through their districts. 

Why stratify? Generally, stratification improves the 
precision of estimates. Usually the stratification variable 
(e.g. enrollment size) is related to the survey variables. 
For instance, teachers in large school districts may have a 
different opinion about discipline than teache^*s in small 
school districts. In surveys dealing with continuous 
variables such as household income or ^.je, stratification 
solves the problem of variability of these characteristics 
by creating homogeneous subpopulations (strata) before 
sampling is done. That is, stratification minimizes 
variances within strata. Stratified random sampling then 
consists of selecting a simple random sample from each of 
the strata into which the population has been subdivided 
(Parel, 1978). The frame would then refer to the lists of 
teachers from the various strata. On the other hand, 
stratified samplin g with pr obabil itv proportional to size 
(ppa) uses a measure of size, such as the number of 

teachers, to determine the probability of selection of a 
school district. 
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Efltimation from Strat if^gd San^pling 

It Is difficult to talk about sampling t^rocedures and 
techniques, without discussing estimation. In determining 
the precision of an estimate > the variance of the mean is 
the key parameter in computing the sampling error. 

To obtain a measure of precision of the estimate of the 
population proportion, an estimate, of the variance of the 
sample proportion is computed by the equation (Parel» 1978) 



or 



where 

L is the number of strata 
h is the stratum number 
N is the population total 

Nh is the number of sampling units in the hth stratum 
n^ is the sample size in the hth strati;im 
Pj^ is the proportion estimate of the population 
proportion in the hth stratum 

(^^J'^'') is the finite population correction for the hth stratum. 

Determination and Allocation of the Sample Size 

In stratified sam:>ling, the first step is the 
determination of the overall sample size (n). The second 
step is allocating n among the L strata. References on 
sampling methods discuss in detail several methods of 
allocation. For instance, when the sampling units N'h in the 
L strata are more or less the same in number, the method is 
equal allocation. VHien the total number of sampling units 
Nh vary from stratum to stratum, sampling is proportional to 
size. 

Thus, in equal allocation. 




in proportional allocation. 
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In other crises i allocation Is neither equal nor 
proportional. For Instance, in cases where one of the 
strata needs special analysis, then one may need a sample 
larger than the size allocated by either the proportional or 
the equal allocation. Whenever such disproportlonal unequal 
allocation is used, adjustments In the overall estimates 
are made by appropriate weighting. 

Samplinic Problems 

Deming (1968) very aptly placed sampling design within 
the context of survey design. In designing a survey the 
basic questions to be answered are: 

o What questions should be asked? 

o What precision is needed? 

o How can the survey best be carried out to provide 
the information desired, with the desired pre- 
cision and no more? 

o What will the survey cost? 

o V/hat do the results^ mean? 

o How can objective measures of sampling errors and 
biases be obtained, so that the reliability and 
meaning of the results can be assessed and methods 
improved? 

Thus, in the design of surveys, the major problems of 
sampling are: 

o Specifying the reliability to be achieved, in 
consideration of the budget and other constraints. How is 
precision going to be meaured? Would one use coefficient 
of variation of IX, 5X, and 15% in some important 
characteristc? Would this measure of precision require 
measurements of the differences between various procedures 
(two or more different ways of stating the questions, 
different approaches in interviewing, different definitions, 
etc.) to achieve better data interpretation? 

Or is one concerned mainly with estimates of 
proportions and would use precision in terms of ±3%, +5%, 
with 95% confidence for the interval of estimate? 

o Designing the survey, (a) so that it will produce 
the desired precision ax, the lowest possible cost and (b) 
with the personnel and other resources available. The 
design should consider the following pointers: 
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(1) Consider minimizing ncn-response \y 
eliminating factors contributing to non-response, such as 
those related to questionnaire clarity, format and length,' 
interviewing style, and time of day. How much non-response 
can be tolerated? How much is enough to impair the survey 
seriously? Can t' ^ first wave of response be improved or 
corrected by interviewing a sample of the people not 
responding at the first interview or not returning their 
questionnaires by mail? 

(2) If necessary, lay out a fer^ alternative sample 
designs to show approximately what the costs will be for 
various degrees of precision; 

(3) Determine the maximum allowable sampling 

error. 



o Appraisin g the precision actually attained in 
several important characteristics, and evaluating the 
differences between the various procedures specified for 
comparison — for instance, an intejrpenetrating sanple in a 
telephone survey, using the same <tuestions. 

Suppose the non-response rates in the two ^samples are 
the same, and responses are similar, one may decide to use 
the cheaper of two methods in the future. On the other 
hand, suppose the responses to some items or the 
characteristics of the respondents in the two groups differ, 
which responses more accurately represent the target 
population? 

It is interesting to remind ourselves that without the 
theory of probabil.^y there would be no way to specify the 
precision prior to the survey and to evaluate it after the 
survey is completed. 



HOW THE STUDY WAS CONDUCTED 



This paper is a critical study of published or reported 
surveys. The "data** in consideration are the information on 
sampling procedures used in the surveys. The numeric results 
of these surveys were outside the primary concern of this 
study. Eight national surveys of public school teachers 
were reviewed, in the area of sampling procea^re. Four of 
these surveys were conducted by major survey organizations 
or agencies, and four by organizations for special purposes 
such as subject-matter-oriented organizations or minor 
organizations with limited goals. 

Exceptions to reviewing only surveys of all public 
school teachers were national surveys conducted by two 
teachers' organizations, where the target poi i lation 
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consisted of public school teachers who were members of 
theli respective organizations* 

In most of these surveys the description of the 
sampling procedure lacked enough details to allow an In- 
depth critical review. In some cases, though, the brief 
description or the way results were reported was sufficient 
to raise que3tlons about the sampling procedure. In these 
instances » the description is quoxed verbatim before the 
critical coDunentarles so that the reader can formulate his 
judguent Independent of this review. Whenever possible, the 
source of the publication was contacted usually by 
telephone. The survey persons w**^*^ usually very cooperatlv<»» 
and many of the clarification qui.^^ions were answered. Ir\ 
additioni we were able to get Information on the practical 
difficulties encountered in obtaining a good sample. 

In one Instance where the survey is still in progress 
and the report had yet to be written, the write-up on the 
sampling design was mailed to us. This was one survey that 
demonstrated how a survey/sampling design was carefully 
delineated and documented! 



CRITICAL REVIEW OF TEN SURVEYS 

The 10 survey organizations are not identified in this 
paper. They are named A to J. In this section, each survey 
organization's sampling procedure will be presented. The 
description will be followed by a critical commentary. 

It will be evident that the survey designs especially 
those of large organizations hinged on how to reach that 
sample of teachers across the country, and achieve an 
acceptable level of precision of ostimates and representa- 
tiveness. The effort to achieve these goals (though not 
explicitly stated as goals) of precision and representa- 
tiveness already deserve some credit, as they would pave the 
way to improvements on design and ^implementation in future 
surveys . 

Most of the surveys depended on some available lists or 
existing data sets compiled within a given year. One list 
used by at least two organizations was that compiled by 
Market Data Retrieval of Westport Connecticut. Two 
organizations on the other hand used stratified multistage 
sampling and teachers were randomly selected at the final 
.•Htage. Ironically, these intricate designs where much can 
be learned were not published; apparently, the details which 
were highly technical were not intended for the general 
reader of the report or publication. 

As d summary, the ten surveys are presented in a matrix 
of the V m surveys across the major characteristics of the 
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sampllrg design. The succeeding sections present these 10 
surveys (not in alphabetical order). 

Survey A 

This survey is a mailed questionnaire survey, A 
"representative sample" of public school teachers was 
obtained from a list provided by Market Data Retrieval- 

"A sample of 2,000 teachers was selected to reflect the 
total national population of teachers." The sample was 
stratified proportionately by region and by teaching 
level • " 

The ruport contained a detailed accounting of the original 
2,000 questionnaires, which resulted in a 41% or 813 
completed returns. Sampling error was not published but, as 
computed for this paper, was + 3.4% at the 95% confidence 
level. 

An effort was made in Survey A to determine representa- 
tiveness that may have been affected by incomplete returns. 
A telephone survey of 100 teachers who had not answered the 
mailed questionnaire was conducted and the results showed 
that the sample of nonrespondents "closely paralleled" the 
responding group both in terms of attitudes and socio- 
economic and demographic characteristics. We analyzed this 
claim of representativeness as reported. In one of the 
attitude qu ^wions, for instance, forty-three percent of the 
mailed-qut^stlonnaire group (with n = 813) answered "yes" 
while 40% of the telephone group (with n = 100) answered 
"yes". How sim..lar are these two groups? Computations of 
sampling error resulted in an interval estimate of 40 to 46% 
and 33 to 53% at the 95% confidence levels for the mailed 
survey and the telephone groups respectively. This analysis 
confirmed tlie report's statement on representativeness, 
although the small sample size of 100 re'sulted In a large 
sampling error. A iest of difference between two 
proportions will produce the same result — that there is no 
significant difference in proportions between the two 
groups . 

It appears that the design was given careful con- 
sideration with respect to precision and representativeness. 
The remaining question pertains to the samplinv frame. How 
representative and up to date is this MDR 1 >t ;hich 
comprised only 68% of the total population ot teachers? A 
minor point, in addition is one statement that may need 
clarification: "The sample was stratified proportionately 
by region and by teaching level." .According to -Lhis 
statement! stratification was mada after the sample was 
drawn. Normally, stratification is done before drawing the 
sample. However, this does not preclude the need to do 
otherwise for practical reasons. If the sample before 
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stratification was a self weighting simple random sample 
where each teachdr had the same probability (sampling 
fraction) of being selected, then a post-sampling 
stratification would still approximately be proportionate by 
region and level. By self weighting san.ple is meant that, 
since the probability of selection was applied to each 
teacher in the list the sample will end up, more or less, 
proportinately representing the various categories into 
which the sample may be broken down. 



S xr vey B 

This survey also selected its sample from the list 
provided by MDR. There were 4,822 in the original sample, 
of which 1846 were completed personal interviews. Agairij 
there is a vague statement here: ''Sample sizes for 
completed interviews were set for each state based on the 
proportion of elementary and s condary public school 
classroom teachers in each sioate.*' If this is so, were some 
selections discarded to meet the known proportion? Or does 
the statement mean that the original sample allocations were 
proportional to the numbers of teachers in the two levels - 
elementary and secondary in each state. The phrase ''based 
on the proportion of elementary and secondary public school 
classroom teachers in each state*' is also not clear, whether 
"proportion" refers to the state's proportion based on thd 
US total; or whether it meant separate proportions of 
secondary and elemeutary teachers. Sampling error was not 
published but as computed for this paper was 4-2*2% at 95% 
confidence level. 

It ras never the purpose of this paper to critique the 
reporting language. However, an appraisal of the design is 
only as good as the understanding of what actually happened 
in the process. 

One indication of representativeness is the 32% male in 
the sample (591/1846) close enough to the known percentage 
of 33.1% in 1981 published by the National Center for 
Education Statistics (Grant, 1984). 



Sur vey E 

The sampling frame was not described nor the source 
named. According to the report, one in 700 K-12 teachers 
were drawn from the population of 2.1 million public school 
K-12 teachers. Two follow-ups resulted in a 45% response or 
1346. Arithmetical calculations showed that the original 
sample size must have been 3000 (1/700 of 2.1 million), and 
the completed return of 1346 was approximately 45% of 3^000. 
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Sampling errors ranged from 1.4% to 22.4% for the 
overall sample proportions. The survey article presented 
the table for the 90% confidence limits for subgroup si2ses 
100 to 1299. This table is shown here as Table 2. 



T«bl€ Z Confidence Limits for Percentages 



AmomuTi mum or moirTAGt roiirrs to u apmd to and suitiactco riOM the oisetvso SAMni 
nucnriAGU to ovqum th m nwcvn conriDciict limits rot tu roruuTicN mceKTAcis 



Sl«» of •ubtroup 

100-199 

200-299 

300-3H 

400-499 

)00-599 

600-499 

700-799 

300-399 

900-999 

l,000-U099 

MOO-1.199 

1,200-1,299 



lOX or 90Z 
5.0 
3.S 
2.9 
2«$ 
2.2 
2.0 
U9 
1.7 
1.6 
1.6 
l.S 
U4 



0bt«rvt4 ptrctncagt nor 

Of 52i 221 at 221 i2i at 121 



7.6 


8.1 


S.3 


5.7 


4.4 


4.7 


3.8 


4.0 


3.4 


3.6 


3.1 


3.3 


2.9 


3.0 


2.7 


2.9 


2.) 


2.7 


2.4 


2.S 


2.3 


2.4 


2.2 


2.3 



The design was apparently a simple random sample from a 
national list. 

The survey indicated that the representativeness of the 
survey sample returns was tested by comparing the "basic 
demographic data with*' the few other known sources of data. 

Though the information on .^sampling procedure is not 
complete I there is every indication that representativess 
and precision were taken seriously into consideration. The 
weakness, again is the sampling frame. How reliable and up 
to date was this sampling frame? 
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Nothing can be said about this survey , because no 
mention was made of the sampling procedure or size. The 
article started by saying that nearly two-thirds of U.S. 
teachers endorse..." The next paragraph described the survey 
as a **nationwidei statistically representative sample of 
U.S. teachers. . . " 

Such statements — as "A clear majority, 62% of teachers 
agree..." or "nearly two-thirJs of the U.S. teachers 
endorse..." — imply exact percentages of the 2.1 million 
teachers. We know it is not so. 

lu another section, the article states tnat "62.7 percent 
of the teachers who responded to our article agree that..." 
This survey did not discuss how the sample was obtained. 
Whether the sample size was 100 or 10,000, it is now known. 
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The article mentioned the availability of a full report 
at a certain price. Since there is a full report which we 
had not examined, there is no commentary on the sampling 
procedure. The percentages , however, were reported in 
sweeping stat^iments which to our judgment was misleading, 
especially if no sampling procedures or errors of estimates 
were discusseid. 



The organization surveyed 252 elementary teachers "from 
all geographic regions of the U.S." 

How the teachers were sampled was not clear in the 
report. To make the published figures work, some guesses 
had to be luade. Thus, the appraisal of this procedure was 
based on the best interpretation one could make of the 
sampling procedure an published. 

According to the report 100 elementary school 
principals were "randomly identified" by the 

. _{name of Association) from a 

ci^ss section of their membership. "Each principal was 
mailed 10 copies of the questionnaire and was asked to 
distribute them to 10 elementary teachers ..." 

There were problems of interpretation, obviously. The 
phrases subject to question were: 

"randomly identified". 

"cross section of their membership. " 

Th€ random selection, if at all it was one, was left to 
persons who may not know or care about sampling. There was 
no mention of instructions given for selection. 

The questions involved are: 

o How random was the principal's selection? 

o How were the principals selected? 

o Does this imply a two-stage simple random sample? 

o If the number of teachers was constant (10) in each 
sample school, what were the probabilities of selec* 
tlon and how were these probabilities taken into 
account in making estimates? 

With regard to the sample size, it appeared that if 10 
questionnaires were given to each 100 principals » then the 
size of the sample was 1,000 to start with. Further 
assumptions or Interpretations were that no follow up was 
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madOf and that 252 was the number of respondents* -a 
reasonable 25% if no follow-up was made. 

The article attempted to show that the sample was 
representative, under the section "Description of the 
Sample" by stating that "teachers were rabher evenly 
distributed from grades K through 6/' the group for whom the 
survey was conducted. 

The problems in giving validity to this survey 
encompass all criteria of a good saraple desian. In summary: 

o The sample size 252 "from all geographic regions" is 
too small tc yield reasonably precise estimates. 

o ''geographic regions" were not defined. 

o A two-stage sample was implied but not explicitly 
described. 

o There was no mention of probabilities of selection. 

o An "even distribution of teachers" across grades K 
to 6 1 regardless of what it meant, does not 
necessarily indicate representativeness. 

o Sampling errors were completely left out and the 
source of sampling frame was not mentioned. 

o The article left the impression that selecting 100 
schools (through principals) and sending 10 ques- 
tionnaires to each principal was, apparently, a 
design for practical convenience, rather than for 
sampling purposes. 



Survey H 

The article indicated that the author wrote principals 
at 500 schools. The principals were given a set of rules to 
draw four teachers at random. 

The 500 schools consisted of 100 private and 400 public 
schools. It was found out that the article was based on a 
report yet to be completed. Further information was 
obtained beyond that given in the article. According to the 
information obtained from the source of the article, MDR was 
a source of sampling frame. However, it was not clear how 
the two-stage sample was drawn — whether the first stage was 
the school, and what MDR had to do with the sample if the 
principals did their own selection. 
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All that can be said is that there was nothing in the 
article that indicated that a well , planned sampling 
procedure was designed. No sampling errors can be computed. 
There is a big question on the representativeness of this 
sample. 

Surveys C and D are the last two surveys to be 
discussed because of their sophisticated stratified 
multistage sampling procedure that required extensive 
description and discussion of details. 

Enough deta Is of the two designs were dibwjssed in 
order to clearly present to the readers two different models 
of stratified multistage sampling of public school teachers. 



Survey D 

The design was a "self weighting stratified, 
disproportionate two-stage cluster with unequal sampling 
rate in the second stage. " This design was based on 
McCall's (1980) description of the process, involving nine 
steps (pp. 273-277). Table 3 (McCall, 1980, p-276) 
illustrates the process. 

Knowing that there were 2,024,000 teachers in the whole 
U.S., it was desired to sample 2,025, or .001, or 1/1000 of 
the total population. With two stage sampling, the 
probability of selecting a teacher is equal to the 
probability of selecting a stratum times the probability of 
selecting a teacher within stratum. 

The public school systems in the United States were 
stratified by enrollment size as shown in table 3. There 
were nine stratft. The source of the list of school systems 
with information on enrollment size and number 
of teachers was the data base of the surveying 
organization supplemented by information from the NCES — 
National Center for Education Statistics, U.S. Department ox 
Education. The total number of systems within each stratum 
is shown in the table. The disproportionate sampling rate 
is shown in column (5) of the table. Notice that all the 22 
systems i' stratum one were selected for the sample giving 
(a sampling rate of 1:1. In the next stratum, 27 systems 
out of the 47 were selected, giving a sampling rate of 
1:1.7. The (unequal) sampling rates for teachers are those 
rates which when multiplied respectively by the sampling 
rate for systems will give 1/1000. This process, with 
disproportionate allocations of school systems and unequal 
sampling rates (for teachers) at the second stage, 
ultimately resulted in equal probability of selecting a 
particular teacher, one in 1,000. In summary, we have the 
following characteristics of this sample design: 
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EXHIBIT 15.2.-DATA USED IN SELECTING A SELF-WEIGHTING STRATIFIED DISPROPORTIONATE TWO-STAGE CLUSTER 
SAMPLE OF CLASSROOM TEACHERS BY USING A LISTING OF PUBLIC SCHOOL SYSTEMS IN THE UNITED STATES STRAT- 
IFIED BY THE NUMBER OF PUPILS ENROLLED. 1979-80 







Total 


Number 


Sampling 


Estimated 


Sampling rate 


Expected 


Probability 






number 


of sys- 


rate for 


number of 


for teachers 


number of 


of selecting 


System 


Stratum limits 


of 


tems in 


systems 


teachers in 


(2nd stage) 


teachers 


a particular 


stratum 


(em'ollment) 


systems 


sample 


(1st stage) 


population 


(1.000^ Col. 5) 


in sample 


teacher 


(I) 


(2) 


(3) 


(4) 


(5) 


(6) 


(7) 


(8) 


(9) 


I 


1 00,000 & over 


22 


22 


1:1 


207.000 


1:1,000 


207 


.00! 


2 


50,000-99.999 


47 


27 


1:1.7 


134,000 


1:588 


131 


.001 


3 


25,000-49,999 


118 


34 


1:3.5 


170,000 


1:286 


171 


.001 


4 


12,000-24,999 


347 


42 


1:8.3 


254,000 


1:120 


256 


.001 


5 


6,000-11,999 


926 


51 


1:18.2 


358,000 


1:54.9 


359 


.001 


6 


3,000- 5,999 


1,856 


45 


1:41.2 


364,000 


1:24.3 


363 


.001 


7 


1,200- 2,999 


3,475 


42 


1:82.7 


336,000 


1:12.1 


336 


.001 


8 


300- 1,199 


4,836 


34 


1:142.2 


169,000 


1:7.0 


170 


.001 


9 


1- 299 


4,004 


32" 


1:125.1 


32,000 


1 :8.0'' 


32" 


.001 


Total 




15,631 


339 




2,024,000 




2,025 


.001 



Source: 

NEA Research: Data used in 1980 Nationwide Teacher Opinion Poll, Februaiy 1 980. 

^ The teachers in the many small school systems selected in the first stage from the stratum were combined as if in a single system 
and the appropriate number of teachers was selected by use of random numbers. 
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o Doaign: "Self -weighting stratified disproportionate 
two stage cluster sampling, with unequal 
sampling rate at the second stage." 

o Target Population: Public School Teachers in the O.S, 



o Sampling Population: First Stage: List of School 

Systems 

Second Stage: Lists of teachers 
within systems 

o Sampling Frame: Same as sampling population 

o Sampling Unit: First stage: School system 

Second stage: Teacher 



o Stratification variable: enrollment size 



o No. of strata = 9 



o Disproportionate selection of school systems 
(first stage) 

o Unequal sampling fraction of teachers (second 
stage) 

o Equfi^l probability of selection of teachers 



Parameters: N 2,024,000 
n = 2,025 

P,j, = 1/1000 probability of selecting a 
teacher 

^s(i) lf...>9 = variable sampHng rate 
for the systems 



The surn/ey being reviewed is in progress; thus the 
figures in this paper were from a similar survey five years 
ago. In the current survey, school systems directories were 
used for the systematic sampling of teachers. 

As discussed by McCall (1980) the self -weighting two- 
stage samples have merits in that the actual sample data 
yield unbiased (or slightly biased) and consistent estimates 
of population parameters without weighting the strata or the 
clusters. 



A word of caution for a very well defined sajnpling 
design should be said in reference to the final stage of 
selecting teachers at random. If this stage is haphazard, 
as when no control is exercised on the procedure, then, 
like, a deck of cards, the whole design, could 
crumble, and the estimates made from the sample data not 
meaningful. Upon further investigation of this process, 
care was in fact exex*ctsed| so that the teachers selected 
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ware truly random. This design should rate as excellent in 
both theory and practice. 

Survey C 

In order to meet the objectives ox the survey, a sample 
of public elementary and secondary schools was drawn and the 
school questionnaire administered to the principals in the 
sample schools. Within the sample schools , a sample of 
teachers was drawn » and a different survey questionnaire was 
administered. 

The sampling frame f pr the selection of the sample 
schools was the Common Core Data" (CCD) 1983*84 universe of 
public elementary and secondary schools. Information on 
this frame allowed for stratification by geography, by size 
and by level of the school. 

The schools were stratified by level: 

Stratum 1 Grades 10, 11, 12 

Stratum 2 Grades 9 and below 

Stratum 3 Ungraded, vocational, special education and 

alternative schools 

Within level the schools were further stratified by region 
and further stratified by the size of the school measured by 
its Fall membership. The report did not give the number of 
regions or the number of size strata. Let R be the number 
of regions and S the number of si^e strata. Then the total 
number of strata was 3 x R x S. Suppose there were four 
regions and three size strata, then the total number of 
strata is 36. 

The probabilities of selection of the schools were 
equal to the square root of the number of teachers at the 
school. 

With the school as the unit of observation, the design 
was a one-stage stratified sampling with pps (probability 
proportional to size) where the MOS (measure of size) was 
the sqaure root of the number of teachers in the school. As 
a survey of public school teachers in the U.S.*-the subject 
of this paper — the design may be described rs a stratified 
two-stage sampling of teachers. The sampling frame at the 
first stage consisted of the list of schools within the 
strata, and the sampling unit was the school. 

At the second stage, a list of all teachers who taught 
at the sample school wa.** the sampling frame > This list was 
obtained from the state education agency, and if not 
available, from the local education agency. 
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These sample teachers were further scratlfled into 
three strata by field of assignment** (1) general or 
elementajry education i (2) mathematics and sciences, and (3) 
all other fields. The sampling fraction was computed for 
each of th^ three t-^acher strata. 

The scheme was somewhat complicated i hence the 
succeeding procedure is reported here verbatim from che 
description of the design i with permission of the author. 
The refetrence used is a design drawn up before conducting 
the survey » hence i the future tense. 

Table 5 give:5 an illustrative determination of the* 
sampling fraction. 



Table 5. Determination of the Sampling Fraction 
(Survey C) 



Teacher 
Strata 


Number of 
Teachers 
from the 
Sample 


Desired Sample 
of Teachers 


Sampling 
Fraction 


Weitfht 


Elementary 


40,000 


4,000 


0.10 


5 


Math & 
Science 


10,000 


2.000 


0.20 


10 


All Other 


50.000 


4.000 


0.08 


4 


Total 


100,000 


10,000 


0.10 





After the sampling fractions have been established a 
probability selection of teachers will be made. An equal 
number of teachers will be selected from each school where 
possible. If the school has fewer teachers than the 
required sample size, then :^11 the teachers in that school 
will be taken in the sample. Otherwise, a probability 
samplti will be drawn following the procedure outlined below. 
Suppose a sample of four teachers is desired in each school. 
To obtain the overall sample size in each teacher stratum 
the sapling fractions will be applied thru the use of a 
weight variable to assign differential probabilities of 
selection. The above illustration has such a weight 
variable included. If school A has 15 teachers » of which 
eight are elementary teachers, two are math/science teachers 
and the remaining five are stratified in the other category, 
then the sampling would proceed in the following fashion. 

Weight up the counts in each stratum by the appropriate 
weight variable (using the weights from the illustrative 
example above), producing counts 8X5:^40 (the count of 
teachers in the stratum multiplied by the stratum weight), 
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2X10s20, and 5X4s20. The total weighted count is 80 
(40+20-t-20r80). The aampling interval is the weighted count 
divided by the sanple sise (80 divided by 4s20). A random 
start between 0 and the sarapline interval is chosen to 
obtain the first selection number; , the sampling interval is 
added to the random start to find the second selection 
number, etc. 



The selection numbers are then compared to the cumulative 
weighted counts to find the number of teachers to be 
selected in each stratum* The application of the procedure 
to determine the number of teachers to be selected from each 
stratum in a school is illustrated in whe example school 
described earlier. See Table 6. 



TABLE 6. School A (Survey C) 



Teacher Listed Weight 
Stratum Teachers . 



Weighted Cum Weighted Sample 
Counts counts Size 



Elem. 



8 



40 



40 



Math & 
Science 

All Other 



2 
5 



10 
4 



20 
20 



60 
80 



1 
1 



Total 



15 



80 



The design proposed two alternative selections of teachers. 
Systematic sampling is demonstrated as follow5, referring tc 
Table 6. 



The design Interval = 
Random start s 5 
First selection = 5 
Second selection number = 
Third selection number * 
Fourth selection number * 



80 divided 4 = 20 



25 
45 
65 



The verbatim description of the design ends here. 

By this systematic sample scheme it was clear that the 
numbers were cpplied to the weighted counts column » and to a 
list of teacher 1 sorted by subject matter as shown in the 
table. Each elementary teacher would then be given a count 
of 5i (the weight) I math and science, a count of 10, and all 
other I a count of 4. 



Thus, the table should look like Table 7. 
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Table 7. Systematic Selection of Weighted List 

(Survey C) 



Teacher 
Stratum 



Weight Serial Numbers of 
the 15 Teachers 



Wei gh ted Sy s tema t i c 
Numbering selection 



Elem. 



Math & 
Science 



10 



All other 4 



1 


1-5 


2 


6-10 


3 


11-15 


4 


16-20 


5 


21-25 


6 


26-30 


7 


31-35 


8 


3t-40 



9 

10 



11 
12 
13 
14 
15 



41-50 
51-60 



61-64 
65-68 
69-72 
73-76 
77-80 



#5 



#25 



#45 



#65 



With simple random sampling, two elementary teachers of 
the eight listed, one math and science teacher of the two 
listed and one of the five listed other teachers were to be 
randomly selected. 

This procedure gave the appropriate allocation for each 
school. As the procedure was applied to the sampled 
schools, the overall desired sample size was achieved. 

The estimation procedure took into account the weight 
producpid by this sampling design. It was beyond the scope 
of this paper to describe this procedure in ihis particular 
Survey . 

One important feature of this design is that the 
objectives of the survey were taken into consideration in 
determining the sample size, which in turn was going to 
affect tl.'e precision of estimate. To determine the 
precision of estimates ahead of time, this survey obtained 
the approximate numbers of teachers by stratum, in the 
universe, from the "1979-80 Survey of Teacher Demand and 
Shortage. " Table 8 shows the approximate universe and 
sample sizes. 
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Table 8. Hnl verse and Sample Size of Teachers (Survey C) 



V*A^11III 
0 w* 01 vUlD 




SamDle Size 


Sampllntf 








Fraction 




on Ann 


1 nnn 

^ 1 uuu 


0 004 


2 Math & 








Science 


245,000 


2,000 


0.008 


3. Oth^r 


1,055,000 


4,000 


0.004 


Total 


2.200,000 


10,000 


0.005 



The coefficient of variation for u proportion, p was 
used as a measure of precision: 



(7) CV(p) = (o-^/np) 

np 

where CV(p) is the coefficient of variation for a 

proportion, p 

D the design effect, 

n is the effective sample slse. 



According to Survey C, the CV's based on a design 
effect of 1.5 and a response rate of 95X for the teachers in 
respondent schools, for various proportions of teachers with 
a characteristic are given in Table 9. 

Table 9. Approximate CV's for Survey C 



Proportion of teachers Estimated approximate 

with a characteristic CV for proportion 



0.75 


0 


008 


0.50 


0 


013 


0.35 


0 


018 


0.20 


0 


.027 


0.10 


0 


.040 


0.05 


0 


.058 


0.02 


0 


.093 



Tables 8 and 9 were lifted from Survey C's paper with 
the author's permission. 

Like Survey D- this design should xate excellent in 
sampling theory. 
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Membership Surveys — I and J 

Surveys I and J were conducted by two teacher 
orgaaaizatlonsy directed only to their respective members. 
Surveys ifhose target population is not the entire public 
school teachers 9 as a whole, or surveys on selected school 
characteristics such as level, or subject matter are beyond 
the scope of this paper. However, these two reports vere 
reviewed to demonstrate the complexity c design and 
difficulty of obtaining a random sample of teachers in the 
case of sampling all teachers where no complete list was 
available. In membership surveys such as I and J, design 
does not present the problems discussed earlier. 

For Survey I, the sampling error was predetermined to 
compute the sample size. The selection was strictly random 
with equal probability of selection of members. Of course, 
like in Surveys A to H, the question of nonresponse presents 
a problem in representativeness. Several follow ups were 
made to maximize returns — thus reaching a 75% response. 

In survey J, the sample size was 800. No men- 
tion was made of representativeness, and of sampling errors. 
The sampling frame was the membership file. Teachers were 
selected by systematic sampling. 

It is clear that compared to the survey of the total 
public school teachers membership surveys could more easily 
minimize nonresponse. Th^have very well defined sampling 
frames — namely the membership list or file. 

Hence, a good sampling design can easily be 
implemented. A sample random sampling design (one-stage) 
las found sufficient for purposes of reaching the teachers 
to be sampled. 

Matrix Summary of the Sampling Procedures of t he Ten Su irveys 

Table 10 is a summary of the features of the ten 
surveys for qualitative comparison. An attempt was made to 
secure independent "ratings" of the eight surveys of public 
school teachers. It was not feasible to ask the intended 
raters to read eight reports for purposes of ratings. 
Secondly, the ratings would not have much validity since the 
description of the sample came in different formats--* 
articles, unpublished reports, unfinished survey reports » 
telephone conversation, personal interview, etc. 

Thus, this tabulated summary would serve as a summary 
to give the reader a birds-eye view of the current surveys 
going on. Table 11 is the rating system to be used but is 
now suggested as a posixble evaluatior: sheet that the reader 
of this paper may ^ant to use to appraise the surveys 

reviewed or tD use as a form to plan and evaluate their o^^n 
survey/sample design. 
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Table 10, Sampling r^ocedures of i National Surveys of Public School Teachers in 1980 to 1985 



SURVEY 



Sampl e 
Size 



Representativeness 



Precision/Sampling Irror Source of Sampling Sampling Design Questionnaire 
Value How to determine Fram* 



099k) 
8B 



41% respcritied. 

NR ^ bias 
resolved 



at 95% CL 
(not re- 
ported) 



Z^( p (1-p )) Market Data Re- 



N large 



Retrieval > 
West port, CN 



"The sample 
was stratified 
prop.by region & 
teaching level" 



Mailed 0. 



1,8<»6 



None cited; but has 
331% male; approx., 
same prop., as pop. 



± 2.2% at 
95% CL (not 
reported) 



-ditto- 



MDR 



N=1.2 mil 



Sample proportional Personal 
to teaching level, Intervii 
for each state; cited 

similarity of data w/ 

reputable survey 

group 



2,800 sch 



10,000 
teachers 



Report not completed sch= CV = 

but sample was de- ;CH-.172 ^ J )^ 

signed to ensure Teachers^ np 

represent «itiveness .008-. 093 (See text) 



(name) = universe 
of public elem. & 
secondary schools 



1^ 



N=2.2 mil 



Design ensures re- 
presentativeness 
but no discussion 
of nonresponse 



Multi stage stra- 
tified sample; 
Level -Regi on-School 
-Teacher. 
Probability selec- 
tion, weighted data 



Varies by At 90% CL 
the values 1 .6kS x S 
of the mean * 
and s 



"From a comprehen- 
sive file of these 
systems" 



Two-stage Prob. Sampl ing Mailed 0 
^'Public School System 

stratified by size 
^Systematic sample 1 in 

1000 ~ combined prob. 

of «olection of teacher. 



1 in 700 
out of 
2.1 mill. 

1 3 ¥6^ 



Demo data com- ± 1.4% to 

pared w/known sources ± 8.3% 
of similar data; 
finding=rep. 



Used table w/ 
Confidence 
limits for prop, 
from 10%-50%. 



Not given 



No description of sampling procedure and related info. 
Simply reported percent results. 



Apparently simple ran- 
dom from a national 
list 



Mailed Q 
2 

follow-ups 



Mailed Q. 



( 
\ 
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Table 10* continued 



SURVEY Sample 1 Representativeness Preclsion/Sampi Ing Error Source of Sampling Sampling Design Questionnaire 
Size Val ue How to aetermine Frame 



252 "From Statement on repre- None None 

all geog* sentativeness but 

regions unacceptable* 
of U.S." 



None 



100 Elem.Princ"ran- 
domly identified 
..."from a cross 

section of their 
membership 



Mailed 0 
No follow-up 



100 private None 
400 public 



Report Report not 
not finished 
finished 



MOR 



Two-stage Hailed 0 

*^"Rand.Samp.of Private 

& public schools 
**Have set of rules for 

princ.to draw k teachers 

at random 







SELECTED TEACHERS BELONGING TO 


AN ORGANIZATION - 






1 


1783 


2 

Representative by 4% e 
Sample Design 


Membership data 
base 


Simple Random from 
comprehensive list. 
Equal prob. selection. 


Mailed 0 
w/follow up 


J 


(800) 


None mentioned None None 


Membership files 


Systematic Sampling 


Mailed 0 



y n Is the effective sample size; the number In parenthesis Is the original sample size. 
2/ NR " nonresponse 3/Coeff1c1ent of Variation 
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Table 11. Please rate each study according to some aspects of sampling by encircling 1, 2, or 3. 1/ When rating, 
bear in mind the relationship between a good/bad sample end the reliability of results.* 



SURVEY Sample Representativeness Precision/Sampling Error Source of Sampling Sampling Design 
Si ze Amount How Determined Frame 



A 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


1 2 


3 1 


2 


3 


B 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


2 


3 1 


2 


3 


C 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


2 


3 1 


2 


3 


D 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


2 


3 1 


2 


3 


£ 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


2 


3 1 


2 


3 


F 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


2 


3 1 


2 


3 


G 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


2 


3 1 


2 


3 


H 


1 


2 


3 


1 2 


3 


1 2 


3 


1 2 


3 1 


2 


3 1 


2 


3 



y Rating: 

1 = Not Acceptable 

2 = Acceptable but results should be evaluated and/or reliability examined. 

3 = Highly Acceptable and theoretically sound. 



ERIC 



31 

BEST copy AVAILABLE 



Page 28 

PROBLEMS 

One of the objectives of this paper was to Identify 
problems associated with the sampling practices prevailing 
In national surveys of public school teachers, and to offer 
solutions to these problems. As gleaned from these surveys 
the problems were: 

1) lack of sampling frame or list of all public school 
teachers from which sample may be drawn; 

2) the assurance that the sample drawn, (or, in effect, 
the respondents in incomplete returns of a survey) is 
representative of the target population of public 
school teachers; 

3) the theoretical connaction between a seunple design and 

the computation of estimates from the results; 

4) the recognition and knowledge of the quantitative 

relationship betveen the samole ciesign and the 
precision or estimates; and 

5) reporting results that demonstrate validity, and 
willingness and ability to sort out the strengths 
and weaknesses of the survey. 

These problems are all Interrelated, hence the 
discussion and proposed solutions do not correspond one-to- 
one to the enumerated problems in this section. 

Discusaior, ^j^j ?r?r ?sed Solntiot^ a 

fi et^pnining Bepresentativcn e aa of f h ? MDR H.^i^ 

"'^'^ comprehensive list of teachers is that 
cZliilt only*l5roS*^;/f f^*T?,.^* ''"^f^'^ Connecticut. The list JSJch 
be?ieen *he Lr2^ L™^? million teachers indicates a large dlscrepanc 
rSlied on this fist sampling frame. Still many surveys 

One survey tested representativeness of incomplete returns (though no 

iSd^aSJSuSLy^i*^ ^''^ comparing demoSraphil.^ocloecSnoSJc 

and attltudinal variables of tne respondents with those of the 

"5? interviewed by telephone, using the same 

a^uSi ''ill continue to be used, a 

K- 2™ r representativeness ought to be conducted. One way would 

fJom S?!fi?5** '•"^i* *^1» "1**^ * drawn directly 

df??« !i 1^«*' according to a comprehensive, multistage 

y**? ^"'^•y' C and D, assuming that the within stratum 
^'^^^ random. Otherwise, the problems oi lack of sampling 
achiJv**; .r«!f?«! J^f''*''!" resolved by designs that eventually 

SatioMl 11^ teachers without necessarily drawing then from a 
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]topr«a«ntatlvttBMs aay also suf far f roa nonrasponse in nail survara 
sinca tha nonraapondanta may ba diffarent from raspondents in tha ' 
variabla(a) bainj studiad. Edgar (1986) conductad a review of studies 
coarracting for nonraaponsa biaa. In har currant work, she is davelopin« a 

c^lLlt "ij^i:.'"" variables for a 

In aurvajrsC and D, tha multistage stratified sampling design resolv.H 
thaaa problema on sampling frame and repraaantativeneas. as well as 
problama of coats. 

* , In stratification by student enrollment in Survey D, there was an 
implicit aasumption that the ratio of students to teachers would be uniform 
across the different enrollment siaes. Therefore, in an earlier similar 
'i'lTT J^^'^^ITf^u^^ heoausB of identical deaign)f 

nSibiirSJ'tfSJh^Si*'?-'***; ^ readily available waa used to determine the 
numbers of teachers in school systems— for which data ware nefe available 
t estimating the number of teacharfSsS^nSolW 
size was used according to the information source for Survey D. 

* *f2^**** study, enrollment size waa again used for 

*Sf?i **^®?,^v? number of teachera within school systems 

which waa available after selecting school systems for the sample, waa used 
in determining the probability of selection. swAe. was usea 

th« T*' ^» selecting the sample. In 

aamn?! m'^^r'**??*^ systems, all systems were included in ?he 

I^IuLa of smaller systems, if all systems were 

a2?i«?!3' "''fi' ''"^r teacher from a school system would be 

MvsSm! tI i!/''^ Z***-^" «0"«aP«nd«nc« "ith a large number of school 

systems, in this design fewer systems were selected; and with the 

?n«SSi^i!v!/ of system entering the formula, more than one 

f?I«ian of 1/1000* " selected systems, to maintain the sampling 

with«I^^Li!w4i?'i*" solved practical problems and minimized costs, 

sif f eS 2 ilJSi!*/:!!^*"!!*" 2?^ reliability. Precision may probably 
SnitS within itSta d«P«ndin« on the homogeneity of the sampling 

Biafig^ gMB<^ h^tm i^sniLSxQ wUd Sample Size in ^t.r.*.^f^ ^A ?«»p]in g- 

In ♦hS*?®'? 'f^^f ^^^^ estimation, the first thing to do after the samples 

5!„!i"*****r* selected is to examine the closeness of the 

52JiIi4S^! V ?J t«*«h«rs drawn from tha strata to the expected number as 

?lSi^^****uf*!^f?; expected sample sizes are shown in column 

JLl ? ! illustrates the stratified, disproportionate, two- 

JiJfri.^'iu"!*? sampling. In the actual implementation of the design the 
f;!*!.^!*.? 2*** »«n»Pl«<* in Survey D were contacted and given uniform 

instructions for systematic sampling, using the sampling rate for the 

second stage (Column (7), Table 3). • The gap between the 
Jil VT!!! d«ta (used in the sample design) were collected and 

r^^y^S,%.; IJl5*" systems were contacted for selecting teachers would of 

gg^g curse produce a discrepancy between the sample size determined at the 
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planning stage (estimated sample size) and the actual number sampled. The 
actual procedure was to give the precalculated sampling interval to the 
person in the school system. Thus, it would not be realistic to expect 
exactly the same number as estimated. However, the respective stratum 
sample sizes should not be too different from the expected. Table 12 
shows the estimated sample sizes by State and by Stratum. At the bottom of 
the table are actual sample sizes by stratum, totalling 1998. Such a table 
as this, which shows the distribution of the nine strata among the states 
would bring the reader to the reality of sampling which is presented in a 
different light from the theory from which its process emerged. 



Table 13 shows x,hB actual total number of teachers N in each stratum, 
the actual stratum sample size n and the prior estimate (n ). This table 
Illustrates the deviation of the implementation from the plan. The 
discrepancy and its percentage as a fraction of the estimate were also 
presented. 



Table 13. Actual and Estimated Sample Sizes in the Nine Strata of Survey D 



Stratum Actual No. Total No. Estimated Discrepancy Per Cent 





of Tchrs 


of Tchrs 


No. of Tchrs 


d 


Discra- 




in the 




in the Sample 




oancy 








-h' 


d=ni,- n^ 
















1 


192 


175,028 


175 


+ 17 


.097 


2 


182 


159,788 


160 


+ 22 


.138 


3 


179 


198,272 


198 


- 19 


.05S 


4 


266 


276,713 


277 


- 11 


.040 


5 


350 


372,147 


372 


- 22 


.050 


6 


323 


348,258 


348 


- 25 


.072 


7 


232 


345.413 


345 


- 13 


.038 


8 


198 


189,558 


190 


+ 08 


.042 


9 


76 


58.873 


59 


+ 17 


.?88 


U.S.: ALL 










Strata 


1.998 


2.124.050 


_ 2,134 


-26 . 


.012 
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Table 12* Distribution of Teachers Sampled by Survey D 
Across Stcte and Strata, 1985 
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The total sample size of 1998 was Just one percent less than the 
estlnated 2|124. By stratai the discrepancies in proportion to the 
estimated sample aise, range from 4X (strata 4i7,8)i to 29X 

oversamplinf in stratum nine. Stratum nine by design was the group of 
school systemFv with the smallest enrollment-**l to 299. It is Interesting 
to note (see discrepancy column, d) that the smallest (strata 8 & 9) and 
the largest (strata 1 and 2) school systems were over sampled i while those 
in the middlv^ (Strata 3 to 7) were undersampled. The authors of this papor 
were not attempting to speculate on the explanation but these are the types 
of information that may be used by the sampling practitioner to improve on 
the next design. 

Sample Design and Estireatlon--Theoretical Problem 



In mo^t of the survey reports reviewed, the sample design was 
relegated to the background. Apparently, the important thing for most 
researchers was the assurance of an adequate, representative and reliable 
sample, or at the very most, an explanation of flaws in this area. Results 
were reported » without consideration of the method of sampling. For 
Instance I in using clusters or strata ere the variances within strata 
homogenous? If not, how would the separate estimation of variances and 
mean, by stratum, be accomplished, and how should they be summarized? 
Books on sampling present a wide range of methods of sample allocation — 
such as equal, proportional, optimum (when cost enters the formula/or 
Nejnnan (when the stratum variance ^^or proportion is expected to vary 
from stratum to stratum (Parel, 1978). 

In order to demonstrate the need for this connection between sample 
design and estimation, let us consider Survey D. McCall (1980, p. 274) in 
discussing sample size determination for stratified two-stage sampling, 
pointed to the problem of lack of adequate preliminary information 
regarding the nature of the population to be surveyed and the expected 
variances to be encountered. In estimating the sample size, one has 
usually no recourse but to make certain assumptions « In the course of 
completing the design, or after a pilot survey, additioi^al information may 
come up and thereby one could make a more realistic determination of sample 
size. After data have been collected, all prior estimates or assumptions 
snould be discarded and estimation made, in accordance with the design. 

To illustr«^te one procedure of mean and variance estimation for each 
stratum after data from the survey have been collected, let us use the 
relevant columns of Table 14. Table 14 shows how to compute the estimates 
of the stratum mean and the stratum variances. 

The illustrative data are as realistic as possible. Thus, the mean 
salary for each stratum was esti::nated from the states' average salaries of 
instructional staff, 1983'-84 (NEA, 1984). Thus, the data across strata 
appeared more homogenous than they probably would be, had the salary data 
for each individual teacher in the sample been available. 
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Table 14. Computation of the Sample Mean and Variance of the Nine Strata 



Stratum 



Total No. 
Sampling 

xh tne 1 
Stratum-^ 



Sample taken 
from the 
Stratum-'- 



Estimate of 
Stratum 
Salary^ 



Est, of Stratum 

variance Stan. 

dev. 



Variance 
of the 
mean 

!hi 

"h 



1 


175,028 


192 


$25,030 


18,443,896 


4,295 


96,062 


2 


159,788 


182 


21,991 


9,421,297 


3,069 


51,765 


3 


198,272 


179 


23,285 


19,331,124 


4,397 


107,995 


4 


276,713 


266 


22,881 


15,187 ''28 


3,897 


57,097 


5 


372,147 


350 


23,341 


14,523, H 


3,811 


4,495 


6 


348,258 


323 


22,752 


8,671,700 


2,945 


26,84 


7 


345,413 


232 


23,037 


11,242,630 


3,353 


48,460 


8 


189,558 


198 


21,949 


15,938,899 


3,992 


80,499 


9 


58,873 


76 


21,144 


8,028,796 


2,834 


105,642 


Total 


N=2, 124,050 


n=l,998 











Data from Survey D 

^Estimated from state averages. If this were an actual computation, 
would be the mean salary of the sample teachers in stratum h, considered as 
estimate of the mean salary of all teachers in that stratum. 

The estimate x^^ of the population mean, x is 
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L 



(8) 




where Xj^ is the estimate of the mean of the 
hth stratum and L is the number of strata. 



Computing from the table, 



^st= $23,063. 

This would be the estimated average salary of teachers in the whole U S 
(illustration only, not fact). 



The estimate of the variance of the sample mean is 



(9) 




or 



(10) 



*(x„) = 



N2 h- 1 



2 '^h^-"'')»h 



where. 



2 



the sample variance in the hth stratum 



N 



h 



n 



]l is the finite population correction (f.p.c.) for the 
hth stratum. 
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Thia f.p.c. may be ignored if it is greater than 
or equal to 0.95, or if rv/ll. is less than or equal to 
0.05 (Parel, 1978). 

Since in national teacher surveys, we are sampling 
from a very large population, would be very 

small — in this example- -approximately 1/1,000. We can 
then Ignore 

(11) (Nj^ - ^h)/^h equation (9). This reduces equation (9) to 

(12) v(x3t) - i k (^h • fh^ ) • 

Using the values in Table 14, we obtain 
v(Xst) = 4,870. 



What is shown here is the connection between 
design and estimation. When estimating the population 

statistic such as the mean, N or the total number of 
sampling units in the stratum enters the computation as 
the stratum weight. Likewise, the variance of the mean 

would be the statistic in computing the 
reliability (or sampling error) of the estimate of the 
mean. 

Thus if the strata in an actual survey are not 
very homegeneous as measured by V(Xfi)--a 
caaracteristic normally expected or desired in 
stratified sampling--and this procedure is ignored, by 
using the unweighted mean of all sampled units in the 
strata, then this estimate would not be accurate. 

For want of better data to illustrate the 
connection between design and estimation, we have used 
the data that were collected for a design involving 
cluster sampling. 

It is at this point that we consider it more 
helpful to distinguish between stratified and cluster 
sampling. 
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If the population consists of groups with great variation In 
their means, It Is usually recommended to divide or stratify the 
population into more or less homogeneous subpopulatlon or strata. 
This design Improves efficiency, that is, minimizes the variance. The 
characteristics of a stratified population usually, is that the 
variance withiq strata is small, while the variance between strata is 
large. One can see, that with this rationale in mind, it becomes 
necessary to make separate stratum estimates of the parameters as we 
have shown, before estimating ^ne population parameters. 

Similar to the strata in stratified sampling, clusters are 
mutually exclusive subpopulatlons which together comprise the entire 
population. Unlike strata, clusters are preferably formed with 
heterogeneous rather than homogeneous elements, so that each cluster 
will be typical of the population (Parel 1978). 

The advantages and disadvantages of stratified random sampling 
and of cluster sampling, as discussed below will be more appreciated 
in the light of the examples given. 

Advantages and D isadvantages of Stratified Random Sampli ng CP&rel. 



The Advantages of stratified random sampling are: 

(1) it is more efficient than random sampling if the population had 
been so stratified that the stratum variance Sj^2 small relative 
to the overall population variance s^. 

(2) it allows for more comprehensive data analysis since information 
is provided for each stratum or subpopulatlon; and 

(3) it is ixdministratively coxivenient 

The disadvantages of stratified random sampling are: 

(1) the stratification of the population may mean the need for 
additional prior information about the population and its 
subpopulatlons ; and 

(2) a separate frame is needed for each stratum. 
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Advent a^ea and Disadvan tagea of Cluster Sampling fPar gl^flTgl 

One advantage of cluater sampling is that there is no need to 
construct a list of elements in the population as one must in random or 
stratified sampling^ the frame for cluster sampling is simply a list of 
the clusters. 

One disadvantage of cluster sampling is that it is no^ so efficient aa 
random or stratified sampling. Thus, adjacent schools and teachers in 
these schools may have more similar characteristics than those distantly 

apart. For cluster sampling to give precise estimates the elements within 
clusters should be heterogeneous with respect to the characteristic being 
measured, so that the sample will be more representative of the population. 

Going back ^o Survey D, without knowledge at thia point, of the 
characteristics ^2 the subpopulationa, it waa not eaay to detennine whether 
the intent waa to atratify or to form cluatera; or whether the deaign waa 
motivated by the feaaibility of aampling or ad&iiniatrative convenience. 
Nevertheleas, the very well defined procedure and the availability of 
comprehenaive liata of school aystem. and teachera within ayatema, aa well 
aa the information on the ntimbera achool ayatema within atrata, making 
estimatea uaing formulas fcr atrui^ified aampling would not hurt. 

The beat interpretation we can give of Survey D ia that ita procedure 
ia that of atratif ied aampling where the atratif ication variable ia 
enrollment aize. "^.len it became apparent that if one conaidera the 
variable aalary, the cluatera of teachera may be homogeneoua with respect 
to salary; and if so, the nine atrata may be conaidered aa cluatera. 

In planning on complex deaigna, one ahould have prior aaaurance tha'* 
the final unit of analyais, the teachera could be aelected randomly to 
continue to enaure repreaentativeneaa. Much of the operational problema 
lie in the final aampling of teachera. For inatance, the achool or achool 
syatem may not furniah a liat, and/or the aelecllon ia left to another 
school peraonnol with vague or without inatruction on random aelection. 
There were caaes from the aurveys reviewed, where no inatruction wa? given. 

When one faila at thia tail-end of the procesa, then the aample deaign 
and ita implementation may be compared to i caatle built from a deck of 
carda where in the laat card at the top topplea the vhole deck. 
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CONCLUSION, IMPLICATIONS, AND EDUCATIONAL IMPORTANCE 



The concern ove*; the use of approriate statistical procedures is not 
limited t.; education-*x^elated surveys. Neter (1986) in his presideiwial 
address at the 1985 Aktnual Meeting of the American Statistical Association, 
discussed concerns over the uses of statistical sampling techniques in a 
wide array of disciplines; the theme in that meeting was Statistics and 
Public Policy. 

It may be small comfort to know, that national surveys of teachers are 
not tho only projects experiencing problems in sample surveys. Johnson and 
Smith (1969), in describing the tremedous impact of sampling procedures 
throughout the modern world pointed to the need of solving problems of 
nonsampling errors and large sampling errors among others. Their book, a 
compilation of papers by experts in the field provided a broader view of 
the theoritical aspects of survey sampling. 

It seems that surveys of the general public like that conducted by The 
Gallup Organization (Gallup, 1985) — "A Gallup Survey on Education" — and by 
the Census Bureau (1980) in its 1980 census sample, had rigorous sample 
designs, practically beyond question in theory and practice and in 

X porting. Gallup' s design of the saunple — '*a replicated probability sample 
down to the block level in the case of urban areas, and to segments of 
townships in the case of rural areas" — took serious consideration of 
reliability and implemented preventive measures that ensured 
representativeness, as well as technical measures that would reduce the 
sample bias. The Census Bureau (1980) on the other hand, took into account 
both sampling and nonsampling errors. Such concepts as sampling unit, 
confidence intervals, standard errors, were deeply entrenched into the 
operational aspects of the design; e.stimation procedures and control of 
nonsampling error were clearly connected with the design. 

Thus, if a certain standard of credibility is being maintained for 
these public surveys, why can*t the same standards be applied to national 
surveys of public school teachers? With educational reforms on the top 
agenda of policy makers on the one hand, programs, laws and regulations and 
otaer educational decisions on the other hand, are rising in equal fast 
tempo. Serious problems arise when persons in charge make decisions, 
formulate policiefi and develop programs based on faulty data and bad 
samples . 

The crucial role of surveys in shaping policies and decisions on 
educational matters cannot be overemphasized. A survey that misleads the 
public on the opinions of teachers, on account of nonrepresentative and 
inadequate samples can do a great deal of harm in making decisions about 
education. 

On the other hand, a survey that truly represents the opinions of ou^ 
educators would have contributed toward the total input into the process : 
determining what is good for the students, the teachers and the school, all 
inseparable in the shaping of the future in education. 

It is expected that this presentation would draw more attention to the 
importance of appropriate sampling procedures in the reliability of 
estimates and representativeness of the sample. For sc survey 
organizations, the solutions offered could be of help i4 mproving their 
O traditional s/unpling approach. 
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Attachment 1 



Pena/Henderson 
AERA, 1986 



How to Determine The Sample Size 


in Estimating a Proportion 


Confidence Level and associated z 




(Framel) 




Confidence Level 


z_ 


90% 


1.65 


95% 


1.96 


99% 


2.58 



Mote 

Confidence Interval is the interval of estimate, measured by z multiplied 
by the standard deviation s. The confide ice level may also be referred to as 
the reliability in percentage values. 



0 
0 



Sampling error = measure of precision; e.g. ± 3%, ± 5%, applied to a 

sample estimate of the population proportion. 

Example: 
(Frame2) 



Given : Desired Con ' ice Interval: 9S'i or .95 
Desired precision: ±3% or ±.03 

Problem : Determine sample size. 



Solution : CI at 95%, is associated with z = 1.96 
given unknown given 
Part A 1.96 x s = .03 

z X stand. dev. = acceptable error 
(Note: For a quick mental computation, use 2 instead of 1.96) 



Therefore: 

(Frame 3) 



,03 



s = desired precision 
^ z 



Sp = .0153 (put this aside) 



ERIC 
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Part B 



The formula for the standard deviation of a proportion , Sp is 



►p '\| 7TrFT 

^ I n ; and fo 



r the variance, is 



Let p « .5 (or S0%) 
1-p ' .5 



n 



Note: .5 yields the highest Sp in a range of values 
from cO to 1.0; hence it is uied in determining 
the sample size n. 



Sp » ,5 X .5 ; 



n « .25 
s. 



Squaring both sides we have 



1 

n » .25 














.0153 (Part A) 


n = .25 0 
(70153)^ 




n ' 1067.9 


or 1068, the required sample size to achieve the 
desired precision of ± 2% 



ERIC 



Comment: (1) The computations above are applicable to sampling from a large 
population. The correction factor (N-n)/N is omitted in Part 
B, since for a large population and a relatively small n, 
the value is close to 1. 

(2) Note that a qoick computation of sample size can be achieved 
using only simple arithmetical calculations in frames 3 and 4 . 

(3) The total information needed in performing the calculations a^e 
in Frames 1 to 4. The text in between are explanatory and may 
be ignored. 
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ATTACHMENT 2 



BIAS AND SAMPLING ERROR 



Concepts and Illustrationa 



Using situations commonl/ encountered in t^A surveys 
these two concepts will be clarified by the illustrations of 
four possible combinations: 

i. Sample is biased and has low precision (large sampling 
error) 

2« Sample is unbiased but has low precision 
3« Sample is biased but has high precision (small sampliaf 
error) 

4. Sample is unbiased and has high precision 

Illuatrations* 

1. Biasedi low precision 

o You know that 40. OX of the membership you are 
surveying are men. 

o Your returns indicate only 17. 4X are men. 

o Your results could be biased (toward the women) i 

especially if the question or item tends to elicit 
responses from women i different from those of men. 

o Further I you have a very small sample so that your 
sampling error at the 95X confidence level is ±» 
8X; your estimates have a loif precision 

o Thus your survey results are not only biased but 
also have a low precision 

Example : 74X favored a given Issue. With a sampling 
error of XBXp you are '*95X confident that the proportion is 
between 66% and 82%.'* 
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Suppose further that the breakdown by sex is given in 
the table below, based on 1000 isample respondents. 



Table 1 Males underrepresentei 

Total Female* Hale 

Yes 737 (74X) 697 (82X) 37 (25X) 

No 263 (26X) 153 (18X) 113 (75X) 

Total 1,000 (lOOX) 850 (lOOX) 150 (lOOX) 

We mentioned earlier that males were under represented. 
You will also note from the table that men tend to amy no 
(75X) while women tend to say yes, on the issue. Therefore, 
the overall proportion sayinf "yes", 74X could be an over 
estimate, because males were under-represented. 

2. Unbiased, low preciaiot^ 

Suppose this time that your sample proportion of men 
and women represent the membership proportion 40-60. The 
table would be: 

Table 2: Correct Proportion of Male members 

Total Female Male 

Yes 592 (59X) 492 (82X) 100 (25X) 

No 408 (41X) 108 (18X) 300 (75X) 

Total 1,000 (lOOX) 600 100 400 (100%) 

Note that there are 400 males, 600 females. Looking at 
the "Total" columns in Table 1 and Table 2, one can see that 
the "yes" proportion in Table 1, of 74X is an overestimate, 
compared to the yes column in table 2, of 59X. 

Note further that even if the 95X C.L. of 1, 8X was 
applied to table 1, the more accurate proportion saying 
"yes", 59X is still below the confidence interval of 68 to 
82 in table 1. 
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3. Bia«ad. hlMh prafl^^i<;>f^ 

Suppose we have a higher precision In Tabl<» 1, ^3%; then 
our estimates (based on a biased saaple) of vhoae saying yes 
would be 74 + 3X or 71X to 77X. The precision has Increased 
or Improved, but note that because of the bias our smaller 
Interval of estimate moves the unbiased 59X further away 
f '^m the Interval. 



4. Unbiased, high praciaion (Ideal combination) 

Suppose we have a higher precision for Table 2, i3%; 
then our estimates based on an unbiased sample is 59 i; Z% or 
56X to 62X. Thus, we can safely assume, with 9S% confidence 
that the proportion of the tot/^l population answering "yes" 
to the question Is between 56f d 62%. 



Sampling Biases 



Sampling Biases (mostly applied to NEA Surveys) 



o Human fa.Mures 

Wanted active classroom teachers in 1965-66, who vmre 
members as of August 1965 but used membership records 
of January 1986 after nonrenewals have been purged. 
Your sampling frame would Qfi^ Include nonrenewals. 

Intentional or nonlntentlonal preferences - e.g. 
in evening calls, accefising only those members who 
answer their phones may result in a higher 
proportion of younger married members. 
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o 



Bias of estimating procedure 



The estimating procedure used has built-in bias 
e.g. erroneous use of the table of random numbers, 
such that each member is not given an equal chance 
of being selected. 

Use of the wrong formula of estimation 



o Bias from failure to randomize starting points in 
S7stematic sampling 

Bias from inadequate definition of the sampling frame. 



Demingy William Edwards; Soma Theory of fiam«l4tttf. Dover 
Publications, Inc.i 180 Varick Street. Mew York, N.Y. 10014i 
1966. 

Jenning, Raymond J. Statistical Survey Tach niQugg^ John 
Wiley and Sons» New York, 1978. 
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BIAS AND SAMPLING ERROR 



Concepts ai*d Illustrations 



Using situations commonly encountered in NEA surveys 
these two concepts will be clarified by the illustrations of 
four possible combinations: 

1. Sample is biased and has low precision (large sampling 

error) 

2. Sample is unbiased but has low precision 

3. Sample is biased but has high precision (small sampling 

error) 

4. Sample io unbiased ancf has high precision 
Illustrations: 

1. Biased, low precision 

o You know that 40.0% of the membership you are 
surveying are men. 

o Your returns indicate only 17.4% are men. 

o Your results could be biased (toward the women), 

especially if the question or item tends to elicit 
responses from women, different from those of men. 

o Further, you have a very small sample so that your 
sampling error at the 95X confidence level is i 
8%; your estimates have a low precision 

o Thus your survey results are not only biased but 
also have a low precision 

Example: 74% favored a given issue* With a sampling 
error of 8%, you are "95% confident that the proportion is 
between 66% and 82%/* 



ERLC 
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Suppose further that the breakdown by sex is given in 
the table below, based on 1000 sample respondents. 

Table 1 Males underrepresented 

Total Female Male 

Yes 737 (74%) 697 (82%) 37 (25%) 

No 263 (26%) 153 (18%) 113 (75%) 

Total 1,000 (100%) 850 (100%) 150 (100%) 

We mentioned earlier that males were under represented. 
You will also note from the tabl<& that men tend to say no 
(75%) while women tend to sav yes, on the issue. Therefore, 
the overall proportion sayin* "yes", 74% could be an over 
estimate, because males were under-represented. 

2. Unbiased, low preeia^ f>n 

Suppose this time that your sample proportion of men 
and women represent the membership proportion 40-60. The 
table would be: 



Table 


2: Correct Propo 


rtion of Male 


members 




Total 


Female 


Male 


Yes 


592 (59%) 


492 (82%) 


100 (25%) 


No 


408 (41%) 


108 (18%) 


300 (75%) 


Total 


1,000 (100%) 


600 100 


400 (100%) 



Note that there are 400 males, 600 females. Looking at 
the "Total" columns in Table 1 and Table 2, one can see that 
the "yes" proportion in Table 1, of 74% is an overestimate, 
compared to the yes column in table 2, of 59%. 

Note further that even if the 95% C.L. of i 8% was 
applied to table 1, the more accurate proportion saying 
"yes", 59% is still below the confidence interval of 66 to 
82 in table 1. 
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